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Abstract 

In this paper, we are interested in investigating the perturbation bounds for the sta- 
tionary distributions for discrete-time or continuous-time Markov chains on a countable 
state space. For discrete-time Markov chains, two new norm-wise bounds are obtained. 
The first bound is rather easy to be obtained since the needed condition, equivalent to uni- 
form ergodicity, is imposed on the transition matrix directly. The second bound, which 
holds for a general (possibly periodic) Markov chain, involves finding a drift function. 
This drift function is closely related with the mean first hitting times. Some \^-norm-wise 
bounds are also derived based on the results in [11]. Moreover, we show how the bounds 
developed in this paper and one bound given in |23] can be extended to continuous-time 
Markov chains. Several examples are shown to illustrate our results or to compare our 
bounds with the known ones in the literature. 
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1 Introduction 

Let $(n) be a (time-homogeneous) discrete-time Markov chain (DTMC) with an 
irreducible and stochastic transition matrix P = {P{i,j)) on a countable (finite or infinite) 
state space E. Denote by = {P^{i,j)) the ra-step transition matrix of $(n). The 
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number d, defined hj d = gcd{n > 1 : P"'{i,i) > 0} for any (then for all) i G E, is called 
the period for $(n), where gcd stands for the greatest common divisor. The chain $(n) 
chain is said to be aperiodic if ci = 1. Obviously, if P{i, i) > for some z G E, then is 
aperiodic. Define ac = inf{n > : ^{n) G C} to be the first hitting time on a set C C E 
and write rriij = Ei[aj]. Note that ma = 0. Let e be a column vector of all ones. Suppose 
that ^{n) is perturbed to be another DTMC $(??,) with the irreducible and stochastic 
transition matrix P. Let A = P — P. Suppose that P and P are positive recurrent with 
the unique invariant probability measure (row vector) vr and u, respectively. Let LI be a 
matrix with equal rows vr. Note that when E is finite, both P and P are automatically 
positive recurrent. We are interested in deriving the perturbation bounds for the difference 
between i/ and vr in terms of A. 

The ^-norm (see [HI [TU]) is introduced as follows. Let V he a. finite function V on 
E bounded away from zero, i.e. in{i(zEV{i) > 0. For a finite measure /i, let fi{V) = 
^•gg /i(i)V^(i) and define its V-noim to be ||/i||y = XlieE 1/^(^)1^(0- Let x be a vector on 
E and define its V-norm as ||a;||y = sup^gg The ^-norm for any matrix L = (Ly) on 
E X E is given by ||i^||y = sup^gg SjgE l-^ijl^(i)- When = 1, we omit the subscript 
V in the notations of \\fi\\v, and ||i^||y. Note that ||/iLx||v/ < and 

< ||y4|| for any pair of matrices A and i? on E x E. 

We now review some known results on perturbation bounds for a DTMC on a finite 
state space. The perturbation bounds mainly include the component-wise bounds for 
\h'{k) — 7i{k)\, k G E and the (measure) norm-wise bounds for ||z/ — 7r|| = X^jge l^(i) "^(i)!- 
The following two formulas 

z/-7r = z/Ai?, (1.1) 

iy-n = uAA*, (1.2) 

derived by [23] and [17], respectively, are fundamental for perturbation analysis. Here 
R = (J — P + n)~^ is the fundamental matrix and A* is the group inverse oi A = I — P. 
The group inverse 5* = (-B.*) of a matrix B is the unique square matrix such that 

BB*B = B, B*BB* = B* and B*B = BB*. 

From [16], we know that A* = R — U. However, A* has more computational advantages 
than R — U. A lot of component- wise bounds (e.g. [3], El El [181 IlZj) and norm- wise 
bounds (e.g. [211 [25]) have been obtained in terms of A*, rriij or ergodicity coefficient. 
For a (possibly negative) matrix B = (Bij), the ergodicity coefficient of B is defined by 
Ai{B) = I supjjgjg ^^,g]g — Bjk\. If Ai(P) < 1, Seneta [21] derived the following 
norm-wise bound 

W--^ < . 1.3 

" "-l-Ai(P) ^ ^ 

Subsequently, Seneta [25] obtained another norm-wise bound 

\\u -tt\\ < Ai{A*)\\A\\, (1.4) 

which holds even when Ai(P) = 1. It was proved by [12] that the bound, given by (1.4), 
is the smallest (best) norm-wise bound. 
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It was pointed out in page 13 of [5] that computation of A* is generally expensive. 
Computing the group inverse A* could be challenging for a large-size finite transition 
matrix. Moreover, the group inverse A* can not be used directly for an infinite Markov 
chain, since itself needs to be well defined. Thus the bounds characterized by A* 
for finite chains can not be simply extended to infinite chains. Let D = (Dij) be the 
deviation matrix defined by Dij = X]^o(-^" ~ "^^^ bound given by (1.3) can be 
extended to infinite Markov chains whenever \\D\\ < oo (see [22]), equivalently, $(n) is 
uniformly ergodic (see Lemma 2.1 in this paper). This bound is very sensitive when Ai(P) 
is close to 1. When Ai(P) = 1, we may consider the ergodicity coefficient Ai(P™) of the 
skeleton chain for some positive integer m (see [20]), which, however, is not easy to 
be determined for infinite Markov chains. Hence it is interesting to look for some new 
bounds which can be expressed in a simple way and can be applied to infinitely countable 
Markov chains. Motivated by these issues, we are focused on deriving new norm-wise 
perturbation bounds for a countable Markov chain in Section 2. Our approach is based 
on ergodicity theory. The conditions, imposed on the DTMCs, are closely related with 
uniform ergodicity ( i.e. ||P" — n|| — )■ 0, as ri — )■ oo). A simple norm-wise perturbation 
bound is derived in Section 2.1 by using a "small set" condition, which is equivalent to 
uniform ergodicity, and which is imposed directly on the transition matrix. This bound 
is obtained by bounding the ergodicity coefficient Ai(P'") in terms of the "small set" 
condition. Please note that this result holds only for aperiodic Markov chains. Hence we 
further present another perturbation bound in Section 2.2 for a general (possibly periodic) 
Markov chain. This bound is given by bounding P — H in terms of the drift condition 
D1(V,C). When the chain is aperiodic, this drift condition is also equivalent to uniform 
ergodicity. As a byproduct, a method is proposed to calculate the ffist hitting times rriij, 
which is different from that in [U]. 

The more general ^-norm-wise perturbation bounds for ||z/ — 7r||y are developed in the 
seminal work of [HI [10] . This topic has also gained much interest in the past decades, see, 
e.g. [HE]- This condition \\R\\v < oo was used by [11] to investigate the ^-norm-wise 
perturbation bounds. When V is bounded, \\R\\v < oo is equivalent to ||P|| < oo. In 
this case, the perturbation bounds for ||z/ — 7r||y do not make much sense, and we are 
more interested in the norm-wise bounds. And, the l^-norm-wise perturbation bounds, 
obtained by [TT], are not explicit enough for being used directly to derive the norm- wise 
bounds. That is why we need to consider the norm-wise bounds separately in Section 2. 
While V is unbounded, the l^-norm-wise perturbation bounds enable us to measure the 
perturbation of the moments of the invariant distribution, which causes essential difference 
from the norm-wise bounds. In Section 3, we derive some V^-norm-wise perturbation 
bounds, expressed in terms of the drift condition D2(V,A,C), for DTMCs based on the 
results in [TT] . 

It is recognized that CTMCs (e.g. [^) are important for modeling real phenomena in 
biology, finance, information, and so on. In the context of queueing theory, many queueing 
models are closely related with CTMCs, for example, the M/M/s/N queue itself is a 
CTMC. Perturbation analysis is not a new topic here, please see [27] for the component- 
wise bounds, see [U [7] for the V^-norm-wise bounds, and see [2T] and [2S] for the norm- wise 
bounds for finite and non-homogeneous CTMCs, respectively. Although a lot of bounds 
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have been developed, it is still worthwhile to develop new and applicable perturbation 
bounds from different aspects. In Section 4, we will show how the perturbation bounds 
developed in Sections 2 and 3 and the bound given by (1.3) can be extended to CTMCs. 
Some conclusions are listed in Section 5, and some related results from [H] are stated in 
Section 6. 



2 Norm-wise bounds for DTMCs 



We first extend (1.1) to a countable sate space. To achieve this, we need to define the 
inverse of the operator / — P + 11. Let ii = {fi : \\fi\\ < oo} be the Banach space of all the 
finite measures. The linear operator / — P + 11 : £i — )■ £i is well defined and its domain is 
ii. Indeed, for any fi G h, fi{I - P + 11) =0 implies /i = 0. Hence R = {I - P + 11)"^ 
is also well defined according to the basic theory about the inverse of a linear operator. 
Note that ||P|| may be finite or infinite. Since ttR = vr and uA = v{I — P + II) — vr, we 
have 

u - Ti = uAR = uA{R-Ii). (2.1) 
If the deviation matrix D exists ( i.e. D < oo), then ^^q(P — H)" = D + 11 and 



(j-p + n) K^(p-n)" = K^(p-n)" (/-p + n) = /. 

\n=0 / \n=0 / 

Due to the uniqueness of a linear operator, we have 

R= {I - P + U)-^ = D + U. (2.2) 
From (2.1) and (2.2), we have 

iy-n = uA{D + U) = uAD. (2.3) 



Note that if the deviation matrix D exists, then the chain must be aperiodic. Based on 
the above arguments, we can conclude that (2.3) holds only for an aperiodic Markov chain, 
while (2.1) holds for a periodic or aperiodic Markov chain. The following proposition, 
most parts of which are known (see [10]), relates the boundness of ||P|| and \\D\\ with the 
uniform ergodicity. 

Proposition 2.1. The following conditions are equivalent to each other: 



(i) ^{n) is aperiodic and \\R\\ < oo; 

(ii) \\D\\ < oo; 

(iii) <l>(n) is uniformly ergodic. 
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Proof. It follows from both Theorem 1 and Corollary of Theorem 3 in [10] that (i) holds 
if and only if (iii) holds. If (ii) holds, then the chain must be aperiodic, and 



ID + nil < \\D\\ + iinii < oo. 



n=0 



Prom (2.2), we have ||-R|| = || Yl^=oi'^ ~^)^\\ ^ (^) holds. If (iii) holds, then from 

Theorem 16.02 in [19], we know that there exist positive constants r < 1 and b < oo such 
that 

||P"-n|| < for"'^ (2.4) 

for any n > 0, which implies that $(n) is aperiodic and \\D\\ < Yl'^=o IK-^" ^ n)|| < oo, 
i.e. (ii) holds. □ 



2.1 A norm- wise bound based on uniform ergodicity 

To derive the main results in this subsection, we need the concept of a small set. Let 
i3(E) be the set composed of all the subsets of E. A set C is called a small set if there 
exist a positive integer m and a non-trivial measure Um on B{K) such that 

P"'it,B):=J2P"'ihk)>iyUB) 

for any i E C and any B G i3(E). For DTMCs on a countable state space, every finite 
set is a small set. It is known from Theorem 16.0.2 in [19] that is uniformly ergodic if 
and only if the whole state space E is a v^-smslX set for some m. 

Theorem 2.1. // the state space E is Um-small for some positive integer m and some 
non-trivial measure u^, then 

\ u-7t\ <^ < — A . 2.5 

" " - z/„(E) - z/^(E)" " ^ ' 

In particular, if there exists some positive integer m such that Ylkm^m{k) > 0, where 
5m{k) := infjgE P™(z, A;) , then (2.5) holds with //^(E) = X]jfc6E^"i(^)- 

Lemma 2.1. //Ai(P'^) < 1 for some m > 1, then 

1 1 prn prn 1 1 

III/ — ttII < 



1 - Al(P'^ 



Proof. Let {$(nm),n > 0} be the m-skeleton chain of $(n). Then the chain <l>(nm) has 
one-step transition probability matrix P*". Since vrP = vr, we have vrP™ = vr, which 
implies that vr is also the invariant distribution of the chain $(nm). Similarly, we know 
that u is also the invariant distribution of the m-skeleton chain $(nm). The assertion is 
obtained immediately by applying (1.3) to both skeleton chains $(nm) and $(nm). 
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Remark 2.1. Following the arguments in Section 2 of f24^ , we can extend this lemma 
easily to more general p-norm \\ ■ \\p, 1 < p < oo (the ip norm on the space of real row 
vectors). Here, we only state this simple case (i.e. p = 1), which was first presented in 
J2D^ using different arguments, to avoid introducing too many mathematical notations. 

Lemma 2.2. // the state space E is Um-small for some positive integer m and some 
non-trivial measure u^, then 

Ai(P™) < l-z/™(E). 

Proof. Since the state space E is i/m-small, we have 

P'"(i,A;) > u^{k), I e E 

for any fixed A; G E. Hence, for any i, we can find a non-negative real number dik such 
that 

which imphes that 

Ai(P™) = lsnpJ2\P^{z,k)-P^{j,k)\ 
< - sup ^{dik + djk) 

= l-^Vm{k) 

km 

= l-z/™(E). 

Proof of Theorem 2.1 By Lemmas 2.1 and 2.2, we have 

1 1 pm pm 1 1 

\\u - ttII < ^ (2.6) 

which is the first inequahty of (2.5). It is easy to derive 

||pm_pm|| < II (p_p) II ||pm-l + pm-2p^...^ppm-2^pm|| <^||A||. (2.7) 

From (2.6) and (2.7), we obtain the second inequahty of (2.5). 

To prove the second part of this assertion, define a set function as follows 

^m{C) = Y,^m{k), CGi3(E), 

fcgC 

where fmik) = z/m({fc}) = Sm{k),k G E. Obviously, the non-negative set function i/^ 
constitutes a non-trivial measure on B{K). Observe that for any A G B{K) 

infP"*(z,A) = z/„(A), 

iSE 

from which, and the first assertion, we obtain the second part of the theorem. □ 
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Remark 2.2. We could have obtained the following perturbation bound more directly at 
the cost of a worse bound that is twice as big as the one given by (2.5). Suppose that the 
state space E is Vm-small. Then Theorem 16.2.4 fT^shows that 



IP" -nil < 2(1 



L-J 



where [—J denotes the greatest integer not exceeding —. From (2.3) and Proposition 2.1, 
we have ^ 

\\u-Tr\\ < \\D\\\\A\\ < ^||P"-n||||A|| < ^S^IIAII. 

n=0 



z/„(E) 



To apply this result, it is helpful to know which kind of Markov chains could be 
uniformly ergodic. When the state space E is finite, an irreducible, aperiodic and positive 
recurrent DTMC $(n) is always uniformly ergodic. For a Markov chain on an infinite 
state space, uniform ergodicity usually requires that the state space should have a "central 
state", which is accessible from all other states in finite time. To see this, we note that 
a positive recurrent Markov chain is uniformly ergodic if and only if for any fixed j G E, 
there is an integer such that infjgE P^{i,j) > -j- > for all n > N. Also, we know from 
Section 2 in [8] that cannot be uniformly ergodic if P is a Feller transition matrix, 
i.e. limj^oo j) = for any fixed j G E. These observations give us some insight into 
uniform ergodicity for DTMCs on an infinite state space. 



Example 2.1. Consider the DTMC on 
transition matrix: 



E 



v 



bo 
hi 
b2 



ai 
as 





ao 
ai 

02 






ao 
ai 



with the following lower-Hessenberg 
\ 



J 



where is the set of all non-negative integers. Suppose that the chain is irreducible and 
YlT=o^k ^ -^^ ^'^ known that the transition matrix of the embedded GI/M/1 queue with 
negative arrivals is of the above structure, in which aj,j > take specific forms such that 
YlkLo^^k < 1- Since Yj'^=Qak < 1, we have doil) = infjgE -P(«, 0) > 1 - YlkLo^^k- From 
Theorem 2.1 we have the norm-wise perturbed bound || A|| , v^°° • '— ' 



Example 2.2. Consider the DTMC on K = Z4. with the following transition matrix 
elements: 

i + 2 > 0, 



Pi 



< 



9, 

P, 
P, 
0, 



if j 
if J 
if 3 
if 3 
else, 



0, z = 0, 

0, i is odd, 

1, i > 1 and i is even, 



where p and q are positive numbers such that p + q = 1 • Calculating the elements in the 
first column of P^, we have do{2) = inf j>o P^(«, 0) = p^. Thus we obtain the perturbation 

□ 



bound ^'ll^ll from Theorem 2.1. 
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To compare our bound with the best one (1.4), we borrow two examples from the hter- 
ature. The first one is from [5], which models the mammillary systems in compartmental 
analysis, and which was used to compare the perturbation bounds in [T3] . 



/ 0.74 


0.11 

















0.15 


\ 





0.689 








0.011 








0.3 













0.4 











0.6 













0.669 


0.011 








0.32 
















0.912 








0.088 



















0.74 





0.26 






















0.87 


0.13 




\ 0.15 





0.047 








0.055 


0.27 


0.478 


/ 



According to [13], we know Ai(y4*) = 11.3352. From this and (1.4), we get the pertur- 
bation bound 11.3352|| A||. Observing the last column of P and using Theorem 2.1, we 
have the slightly bigger bound g-^||A|| = 11.3636|| A||. 

The second one is from [16], whose transition matrix is given by 



P = - 
4 



/ 2 2 \ 
2 2 
2 10 1 
Villi/ 



According to [16], A* is given as follows, from which and (1.4), we obtain the perturbation 
bound 1.5512||A||. Computing P^ (given below), we obtain from Theorem 2.1 the bound 
3.2||A||. 



A* 



1083 



/ 265 -61 

-96 300 

-115 -137 

\ -210 -156 



-96 -108 \ 

-96 -108 
246 6 

-210 576 / 



P' = — 



1 

16 



/ 8 2 4 2 \ 

4 6 4 2 

3 5 7 1 

\ 5 4 5 2 / 



2.2 A norm-wise bound based on a drift condition 

We have known that a uniformly ergodic Markov chain is necessarily aperiodic. The 
bound given by Theorem 2.1 holds only for aperiodic chains. In this subsection, we will 
make use of the formula (2.1) and the following drift condition to derive a norm- wise 
perturbation bound for a general Markov chain which is possibly periodic. 

T)1{V,C): There exist a bounded non-negative function V and a finite set C such 
that 

EP{hjMj)<v{z)-i, z^c, 

i6E (2.8) 
V{i) = 0, ieC. 

It is well known (e.g. [2]) that an aperiodic and irreducible Markov chain is uniformly 
ergodic if and only if its transition matrix P satisfies the drift condition. As will be shown 
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in the following proposition, this drift function is greater than or equal to a sequence of 
the first hitting times. A more general form of the following proposition is proposed by 
Theorem 2.1 in [T5] . 

Proposition 2.2. Let C be any fixed finite set in E. The function V, defined by V{i) = 
Ei[ac],i € E, is the minimal non-negative solution to (2.8), and V satisfies (2.8) with 
equality. Note that the minimal solution means that if there is another solution V , then 
we always have V{i) < V{i) for any i G E. 

Theorem 2.2. Let Iq he any fixed state in E. If P satisfies D1(V, C ) for C = {io}, then 




(2.9) 



and 




(2.10) 



Proof, (i) Let g be an indicator function on E given by 



if i = io, 
else i io- 



Define a measure a on B(E) by 



"(j) = P{ioJ)J G E and 



a 



(A) = ^a{i),Ae B{E). 



Let T = (Tjj) be the matrix given by 




For any e > 0, define the sequence {V{i),i G 
satisfies (2.8), we have 



E} by V{i) = V{i) + e, i 



G E. Since V 




for any z G E, which follows 



1 



(2.11) 



It is easy to derive 
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Hence the condition in Theorem 2 in [TT] (see condition (ii) in Proposition 6.1 in the 
Appendix) is satisfied. It follows from (6) in [11] (see (6.1) in the Appendix), we have 



R-u = u 7r^r"eJ-^r" ) +^r"(j-n), 



n=0 



n=0 



n=0 



Using (2.1) and the fact that AH = 0, we have 

oo 

u -71 = uA{R-U) = z/A^T"(/-n). 

From (2.11), we obtain 



n=0 



^T'^(j-n) 

n=0 



< 11/ -ni 



oo 

n=0 

oo 

n.=0 



n=0 



< 2[supV{i) 



Since e can be given arbitrarily, we have 



5^T"(/-n) 



n.=0 



< 2 lim sup l^(i) + £ <2 supV(i) 



(2.12) 



(2.13) 



The perturbation bound (2.9) follows from (2.12) and (2.13) immediately. 

(ii) From Proposition 2.1, we know that supjgEmjjg < supjgE\^(i) < oo, and that 
the sequence of {mjjQ,i G E} also satisfies the drift condition. It is well known that 
for an irreducible Markov chain, sup^^^mu^^ < oo for some state io ^ E if and only if 
suPigE'^Mo < °° ^0 £ E. By (2.9), we have 



k - 7r|| < 2 ( supmiio 



|A||, 



(2.14) 



□ 



which follows the bound (2.10) since the state zq is taken arbitrarily. 

To apply this result, a key point is to find a drift function, which is the usual way 
to verify ergodicity in the context of ergodic theory. For finite Markov chains, we can 
solve (2.8) with equality for C = {io}, i.e. to compute the first hitting times ma^. This 
is always feasible since it is equivalent to solving a finite system of linear equations. This 
way of computing rriij is different from the one proposed in [5], where rriij is calculated 
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in terms of a g-im/erse G oi I — P (i.e. (J — P)G{I — P) = I — P). For infinite Markov 
chains, we do not have such a general procedure. However, we can make some suggestions 
on this. One way is still to solve (2.8) with equality by making use of the structure of 
the transition matrix and the "minimal nonnegative" property, which will be illustrated 
by the following Example 2.4. The other way is to construct a drift function such that 
(2.8) holds with inequality instead of strict equality. As will be shown by the last two 
examples at the end of this section, it is possible to construct a simple constant solution 
for special models. 



Example 2.3. Consider the DTMC on E := {0, 1, 
matrix: 



/ 


Co 


bo 


■ 


■ 












Cl 


bi ■ 


■ 













0,2 


C2 ■ 


■ 
















■ 


■ Cn-2 


bn-2 













■ 




Cn-1 


bn 


\ 








■ 


■ 


an 





n} with the following birth-death 
\ 



J 

where the coefficients ai,bi and Ci are such that P is stochastic and irreducible. Note that 
P is periodic with periodicity d = 2 if Ci = for all < i < n. For any fixed j G E, 
we define the drift function V by V{i) = mij,i G E. Substituting the value of P{i,j) into 
(2.8) with equality and inducing on m gives 



V{m) - V{m + 1) 



'-[V{m-1) -V{m)] 



A:=0 



for any m, < m < j — 1, where V{—1] 
given by 

/i(o) = i, m = 

Summing over m from i to j — 1 yields 



0, and the sequence {fi{k),0 < k < n} is 



bn - ■■h 



'n-l 



Ol ■ ■ ■ ) On 



l<k<n. 



rrii 



bmfJ'im] 



E/i(A:), i<j. 



(2.15) 



k=0 



Then we consider the case of i > j. Substituting the value of P{i,j), we have 

V{n) - V{n-1) = — , 

On 

and 

V{m + 1) - V{m) 



^iV{m)-V{m-l))-^ 



j + l<m<n — 1, 



where V{j) = 0. Solving these equations gives 

i-l 



rrii 



m = E 



1 

[ bml^im) 



J2 -"W' J <^<n- 



(2.16) 



=m+l 
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From (2.10), (2.15) and (2.16), we obtain the perturbation bound 

where we make the convention that Y12=n^k = Ylk=o^k = for any sequence of at. □ 

Example 2.4. Consider the DTMC on K = Z4. with the following transition matrix 
elements: 

' 1, ^ = 0,J = 1 

Pi, ^>i,j = o 

Qi, i>l,j = i + l 
0, else. 

where pi and qi are positive numbers such that pi + qi = 1. Define the drift function V by 
V{i) = miQ,i G E. Taking Iq = in (2.8) with equality, we have 

Since V is the minimal and non-negative solution, we have 

n n—1 ^ n— 1 ^ 

vil) = sup n E Tv = 5Z rr ^2-^^) 

Hence miQ,i > 1 are obtained by (2.17) and (2.18). In particular, for Pi = p and q^ = 
q = 1 — p, we have rriiQ = — i for any i >1. Using (2.9), we obtain the perturbation 
bound \. □ 

To show how to find a drift function such that (2.8) holds with inequality, we consider 
Example 2.1 and Example 2.2. For both examples, we choose zq = 0. 

For Example 2.1, we let V^(0) = and V{i) = Then for i > 1 

f:P{^,J)V{J) = -r^^^ = < V{^) - 1. 

By (2.9), we obtain the perturbation bound -p^^J^-^^^H A||. This bound is worse than 
the one given by Theorem 2.1. 

The transition matrix P in Example 2.2 is modified into a periodic one with periodicity 
c? = 2, by changing two elements P(0, 0) = p and P(0, 1) = q into and 1 respectively, and 
keeping all the other elements unchanged. For any fixed positive number A, let V{0) = 0, 
V{1) = ^ and V{{) = ^,i > 2. Then we have 

f if g=i^-l = ni)-l, ^ = 1, 

J2 Pihj)V{j) =1 i±^g < if - 1 = V{z) - 1, z > 2, 2 is odd, 
i=o [ ^1 = ^ - 1 = ^(0 - 1> i > 2, z is even. 

Letting A J, 0, we obtain the perturbation bound ^||A|| from Theorem 2.2. 

12 



3 y-norm-wise bounds for DTMCs 



To consider the norm- wise bounds for DTMCs, we need to change (2.1) into 

oo 

i/ = 7r(/- A(i?-n))-i = 7r^[A(i?-n)]", (3.1) 

n=0 

under the assumption that ||A(i? — n)||v < 1. The ^-norm-wise bounds were first in- 
vestigated by [H] through a detailed analysis of the boundness \\R\\v and an explicit 
expression of i? — 11. Based on Corollary 2 in [TT] (see Proposition 6.2 in the Appendix), 
we obtain the perturbation bounds for ||z/ — 7r||y in terms of the following drift condition. 

D2(V", A, b, C): There exists a finite function V bounded away from zero, some finite 
set C, and positive constants A < 1, 6 < oo such that 

J2Pihj)VU) < XV{^) + bIci^), i G E. (3.2) 

Corollary 3.1. Let iq be any fixed state in E. Suppose that P satisfies D2(V, X,b,C) for 
C = {to}. 

(i) Let c = 1 + ||e||v/||7r||^/. // ||A||y < then <l>(n) is positive recurrent and 

II II ^ c||7r||y||A||v/ 
1 — A — c|| A||y 

(ii) IfV>l and \\A\\v < b+i^x ' ^^^'^ '^(^) ^■^ positive recurrent and 

II II M''+l-A)||A||v 
"''""""-(1-A)=-(1-A)(t+1-A)||A||.- 

Proof, (i) Let T = (Tj^) be exactly the same matrix as that defined in the proof of 
Theorem 2.2, i.e. T is formed from P by changing P{iQ,j),j G E into Os and keeping the 
other elements unchanged. Obviously, all the three conditions of Corollary 2 in [11] are 
satisfied. So we have the first assertion immediately from Corollary 2 in |llj . 

(ii) Multiplying both side of (3.2) by 7r(i) and summing over i, we obtain 

5^7r(2)5^P(2,j)nj) < MV) + blTito). 

Using the invariance of vr ( i.e. irP = it) derives 



A - 1-A 

We obtain the second assertion immediately from the first one, by replacing ||7r||y and c 
in the condition of (i) and in (3.3) with their upper bounds and 1 + respectively. 
□ 
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Remark 3.1. (i) It seems impossible to define a measure a and a function h so that 
the single point set {io} in this corollary can be changed into a finite set C. (ii) The 
second bound is completely dependent on the parameters A, 6, F and ||A||y at the cost of 
decreasing the accuracy of the bound. To investigate a specific model, we should try to use 
the first assertion whenever niV) can be computed. 

Remark 3.2. This drift condition D2(V, X,b,C) is sufficient and necessary for an ape- 
riodic chain $„ to be V -uniformly ergodic, i.e. ||P" — n||y — )■ as n — > oo. When 
V is bounded, the condition D2(V, X,b,C) is theoretically equivalent to the condition 
D1(V,C). However, it is much harder to decide the former, since it involves two more 
parameters. That is the reason why we choose the latter to derive the norm-wise bound 
in Section 2. An aperiodic chain $„ is geometrically ergodic but not uniformly ergodic 
if and only if P satisfies D2(V, X,b,C) for an unbounded function V . The V -norm-wise 
perturbation bounds really make sense when V is unbounded. 

4 Perturbation bounds for CTMCs 

We now show how the discrete-time results developed thus far may be lifted through the 
/i-approximation chain to obtain analogous results for CTMCs. 

Let $t be a CTMC on a countable state space E with an irreducible and conservative 
intensity matrix Q = (Qij). Throughout this section, we assume that Q is uniformly 
bounded (i.e. supjggQj < oo, where Qi := —Qu). Let P* = {P^{i,j)) be the correspond- 
ing unique Q-function. It is known that P* = e*^* = ^^r~- suppose that $f is 
positive recurrent with the unique invariant probability measure vr. The matrix Q is per- 
turbed to be another irreducible and uniformly bounded intensity matrix Q = (qij). The 
corresponding Q-process $t is assumed to be positive recurrent with the invariant prob- 
ability measure u. Let A = Q — Q. We are interested in investigating the perturbation 
bounds for u — n in terms of A. 

We now introduce the /^-approximation chain (e.g. [2]), which is a crucial technique 
adopted to extend the perturbation bounds from DTMCs to CTMCs. For the uniformly 
bounded intensity matrix Q, let h < (supj^g Qi)~^ be the length of the time discretisation 
interval. The transition probabilities P'^{i,j) for $(t) have first order approximations 
Ph{i,j) = {I -\- hQ)ij, i,j G E. The matrix P^ is stochastic, irreducible and aperiodic. 
The chain <l>/t(n), with P^ = {Ph{i,j)) as its one-step transition matrix, is called the 
/z-approximation chain of $(t). Define D = (Dij) = f^{P^ — Il)dt to be the deviation 
matrix of $f and let Dh be the deviation matrix of Ph- We have the following basic 
relations between $(t) and $/i(n): 

(i) ^(t) and ^hin) have the same invariant probability measures; 

(ii) ^(t) and ^h{n) are equivalent in positive recurrence, geometric ergodicity, and uni- 

form ergodicity ( see, e.g. [HI [26]); 

(iii) D = (see, e.g. H). 
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4.1 Norm-wise bounds for CTMCs 



To consider the perturbation of the invariant probabiUty measure for a CTMC, we can 
equivalently consider the perturbation of the invariant probabihty measure for its h- 



approximation chain. For any h < 



-1 



let Ph and Ph be the 



max{supigE Qi, sup^gE Qi} 

/^-approximation chains of $t and $4, respectively. Since Ph is aperiodic, can be used 
to characterize the perturbation bounds for the /i- approximation chain, which enables us 
to investigate the perturbation bounds for CTMCs only in terms of the deviation matrix 
D. 

The following drift condition, which is sufficient and necessary for to be uniformly 
ergodic (i.e. ||P* — n|| — ?■ 0), can be found in Chapter 6 of [2]. 

Dl'(y, C): There exist a bounded non-negative function V and a finite set C such 
that 

iGE (4.1) 

V{i) = 0, ieC. 

We are now in a position to state the norm-wise perturbation bounds for CTMCs. 
Theorem 4.1. Let io be any fixed sate in E. 

(i) // the Q -process $t is uniformly ergodic, then we have 

< Pllll^ll- (4-2) 

(ii) IfAiiQ) > 0, then 

where Ai(Q) = |infij iQu - Qji\ + \Qij - Qjj\ - Y.s^i,s^j \Qis - Qjs\ 

(iii) Let 6k = infi^kQik- IfJ2keE^k > 0, then $t is uniformly ergodic, and 

< ^ \ l|A||- (4.4) 

(IV) // Q satisfies Dl '(V, C ) for C = {io}, then we have 

-7r\\ <2 (supV{i)] ||A||. (4.5) 

Proof, (i) Since $(t) is uniformly ergodic, $/i(n) is also uniformly ergodic. It implies from 
Proposition 2.1 that \\Dh\\ < 00. From (2.3), we have 

u-TT =u-Ah-Dh = lyAD, (4.6) 

where = Ph — Ph- We then have (4.2) from (4.6). 
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(ii) Since Ai{Q) < 0, we can choose small enough h such that 



1 

- sup 



\1 + hQii - hQjil + \hQij - 1 - hQjjl + ^ h\Qis - Qj 



1 

- sup 



2 - h\Qji - Qii\ - h\Qij - Qjj\ + ^ h\Qis-Qj 
1 - /lAi(Q) < 1. 



3S\ 



(4.7) 



Hence we have (4.3) from (1.3) and (4.7). 

(iii) Choose small enough h such that h < ttt^ ^- Then we have 

miPh{i,k) > h5k 



for any i G E, from which and (2.5), we obtain 

1 



— TT < 



|A 



lAII. 



(IV) Since Q satisfies D1'(V, C) for C = {io}, we can obtain 

Y,Q^JVU)<->^Vi^), t^to, 

JGE 



(4.8) 



where A = ^ y and ^(i) = V{i) + e for any i G E, where e is an arbitrarily given 

positive number. Transferring (4.8) to the /i-approximation chain gives 

jGE 

Similar to the proof of Theorem 2.2, we can obtain the bound (4.5). □ 

Remark 4.1. If the state space E is finite, then Q is uniformly bounded and the Q-process 
is uniformly ergodic. Hence all the conditions (i)-(iii) hold automatically . 



4.2 y- norm- wise bounds for CTMCs 

We will use the following drift condition, which is equivalent to V^-uniform ergodicity for 
a CTMC, to find the l^-norm-wise bounds for CTMCs. These bounds parallel to the ones 
in Corollary 3.1. 

D2'(y, A, 6, C): There exists a finite function V bounded away from zero, some finite 
set C, and positive constants A, 6 < oo such that 

Y^QijVU) < -Ar(z) + bicii), I G E. (4.9) 

jGE 
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Theorem 4.2. Let io be any fixed state in E. Suppose that Q satisfies D2'(V,X,b,C) 
forC = {to}. 



(i) Let c = 1 + ||e||y ||7r||y. // ||A||y < then <I>(t) is positive recurrent, and 



c||7r||y II A| 
A-cllAII 



(4.10) 



(ii) IfV >l and \\A\\v < then 



6(6 + A)||A| 



(4.11) 



A3_A(6 + A)||A||y" 
Proof. Since Q is bounded and Q satisfies D2'(y, A, b, {io}), we have 

J^PhihjWij) < (1 - \h)vii) + bhi{,^yit). 

The bounds (4.10) and (4.11) foUow from (3.3) and (3.4), respectively. 
Remark 4.2. Suppose that Q satisfies D2'(V, X,b,C) for V > 1 and C = {io}- From 



□ 



then V 



(3.1) and the above arguments, we know that z/||A||y < - or ||A||y < 
TT ^^QfAD]*^. In particular, let Q = Q + eG. Assume that Ge = and \\G\\v gi < oo. 



A2 

b+X> 



Then for any e such that e < — or e < 



we have v = n Y.n=o ■ ^^^^ 



result is better than Theorem 2.1 in since the convergence domain of e for this series 
expansion is more computable and accurate. 

Example 4.1. Consider the CTMC on E = Z+ with the following intensity matrix: 

( ao fli 0,2 03 ... \ 

bo bi 62 h 

60 bi 62 

bo bi 



Q 



\ 



J 



where {aj,i G 1j^}and{bi,i G Z+j are two sequences of real numbers such that Q is stable 
and conservative. A Markovian queue with batch arrivals is a particular case of this chain. 
For a sequence of real numbers {ck, k G define C{z) = Yl'^o ^^z^ ^e the generating 
function, and let (pc be the radius of convergence of G{z). It is known (see, e.g. fT4\l ) 
that is ergodic if and only ifYl'h=o^^k < 00 and B'{1—) < (i.e. Yl'h=i^bk+i < bo). 
Suppose that this chain is ergodic, (p = min{0yi, 0^} > 1 and < B{(j)B) < 00. Define 
p = sup{z : B{z) < 0}. Since B'\z) > 0, B{z) is a convex function in [0,p]. Thus the 

function — is continuous in [l,p]. Hence A := max < — : 2; g [l,p] [ can be attained 



at some point Zo G [l,p]- Furthermore, we have Zo > 1 and A > 0, since B{\) = and 
B'{1-) < 0. Let V{i) = > 0. We have 



-B{zo] 



Zo 



V{i) = -\V{i), i > 1, 
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and Ylj^QojV^j) = A{zq) < oo. Thus the equality in (4-9) holds foriQ = 0, A = A and 
b = A{zq) + A, which implies vr(\^) = b/X. By (4-10) or (4- 11), we have 

A3 - A(6 + A)||A||y 

In particular, let ai = a, ai = 0,i > 2; bo = fi, b2 = cr and bi = 0,i > 3. Then 
the chain becomes the well-known M/M/1 queue. is positive recurrent, i.e. 

a < jj,, then we have zq = , Vii) = Zq, A = — ^JJ^f and b = fi — yfjio . Hence 

the V -norm-wise bound can be given explicitly by (4.12). □ 



5 Concluding remarks 

Two new perturbation bounds for DTMCs are derived by giving computable values of ^ 
in — 7r|| <£||A||. The l^-norm-wise perturbation bounds are also considered. These 
bounds developed for DTMCs are further extended to CTMCs. If a norm-wise bound 
is such that ^||A|| > 2, then the bound is entirely useless, since we automatically have 
— 7r|| < 2. Hence ||A|| is usually assumed to be small. 

A DTMC $(?7,) is said to be strongly V^-stable (strongly stable for V = 1), if every 
stochastic transition matrix P in {P : ||A||y := ||P — P||y < e} has a unique invariant 
probability measure v such that ||z/ — 7r||y— T-OasjlAHy— t-O. It is known from Theorem 
1 in jlT] (see (i) of Proposition 6.1 in the Appendix) that $(?t,) is V^-stable if and only 
if ||-R||v < oo. It can be easily seen that <^{n) is strongly stable under the condition in 
Theorem 2.1 or Theorem 2.2, and that <1>(?7.) is ^-strongly stable under the condition in 
Corollary 3.1. 

Similarly, we can define strong stability for CTMCs. A chain $(t) is is said to be 
strongly l^-stable (strongly stable for = 1), if every conservative intensity matrix Q in 
{Q : II A II y := HQ — Q||y < e} has a unique invariant probability measure v such that 
\\^ — T^\\v — as ||A||y 0. Obviously, $(t) is strongly stable under the condition in 
Theorem 4.1, and $(t) is V-strongly stable under the condition in Theorem 4.2. 

Currently, we only consider perturbation bounds for CTMCs with the uniformly 
bounded intensity matrices. When the intensity matrices are unbounded, the /i- approximation 
chain method can not be used, and the bounds given in Section 4 may fail to hold. It 
is meaningful to know which bounds still hold. To investigate this issue requires some 
different methods, which is a topic for future research. 



6 Appendix 

The following proposition, taken from Theorems 1 and 2 in [11], are important for inves- 
tigating the strong stability and perturbation bounds in l^-norm. 
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Proposition 6.1. Let V be a finite function bounded away from zero. The DTMC $(n) 
is strongly V -stable if and only if either of the following condition holds: 



(i) ||P||y < oo and \\R\\v = - P + Uy^\\v < oo. 

(ii) (1) \\P\\v < oo; (2) There are some finite non-negative measure a and some non- 

negative bounded function g on K such that T = {Tij) is a non-negative matrix, 
where Tij = P{i,j) — g{i)oi{j); and (3) there exist positive numbers A < 1 and 
m > 1 such that T'^V^x) < XV (x). 



The following Proposition is taken from formula (6) and Corollary 2 in |11] . 
Proposition 6.2. Suppose that condition (ii) in Proposition 6.1 holds. Then we have 

i? - n = n [tt{i - TY^ei -{I- T)~^] + (/ - T)~^(/ - n), (e.i) 

where (/ — T)^^ = X^^o^"- Furthermore, if ||A||y < with c = 1 + ||e||y ||7r||y, then 

c||A||y||7r||y 



1-A-cllAI 
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